As the state of the art of digital signal technology advances, related technologies such as digital image processing experience corresponding growth and benefits. For example, the development and proliferation of facsimile communications allows images to be encoded into digital signals, transmitted over conventional telephone line, and decoded into a close representation of the original images. Image data are also digitally encoded for ease of storage, modification, copying, etc. As is common experience with growing technologies, the field of digital image processing is also experiencing problems with applications in new areas.
Problems in the area of digital image processing (coding) relate generally to achieving a balance between acceptable image distortion and bit rate transmissions. In order to increase the efficiency and therefore the usefulness of digital image decoding schemes, the coding system must provide a coded set of image data that is more efficient to store, transmit, etc., than the original image data and must reproduce a decoded image with some minimum level of quality. However, the conversion of relatively high bit rate image data to lower bit rate data virtually always entails a loss of image quality.
One straightforward method for coding an image is to create an artificial grid over the image and to assign a value to each grid space representing the color of the original image at that grid space. If the grids are made small enough and the values represent a large enough range of color, then the image may be encoded and decoded with small image quality degradation. For example, display screen images are made up of an array of pixels, i.e., picture elements. On a black and white screen, each pixel has a value of one or zero representing the on/off state of the pixel. In a one-to-one bit-to-pixel coding scheme, each pixel value is represented as a 1 or as a 0 and the entire screen image is encoded. The result of the encoding is an array of binary values. To decode the image, the array values are translated into a screen image having pixels on or off in the same order in which they were originally encoded.
If the image is comprised of more than two distinct colors, then more than a 1-bit code must be used to represent the pixel values. For example, if four distinct colors are present in the image, a 2-bit binary code can represent all of the values. If the image includes 256 distinct colors, a 8-bit binary codes is required to uniquely represent each of the color values. The memory requirements for such coding schemes increase as the number of distinct colors in the image increases. However, with high bit rate coding schemes, the quality of the image that results from the decoding of the digital data will be good as long as the digital image transmission or recovery from storage is successful.
When an encoding scheme converts a single value, i.e., a single value representing a pixel, into a coded value, the scheme is commonly referred to as scalar encoding. A vector encoding scheme is similar to a scalar scheme except that input vector data rather than scalar data are encoded. With vector data, more than one dimension of the pixel can be described. Vector encoding schemes are useful for image coding when the image is made up of elements having both color and shading characteristics, i.e., intensity, as well as for the coding of other analog data or very high rate discrete data. For example, an image may be comprised of pixels having 8 colors, each of which can assume 4 shades. Each of the pixel values could then be represented by a 2-dimensional vector. The first vector field is a 3-bit binary value for color; the second vector field is a 2-bit binary value for shade. Thus, in order to exactly encode the image, each vector would require 5-bits of data to represent a single pixel value.
To reduce the size of the encoded digital image, bit-to-pixel ratios other than one-to-one or many-to-one are used. A lower bit rate reduces the amount of digital image data that must be stored, transmitted, etc. A number of quantization schemes have been designed to control data encoding and decoding in order to produce a lower bit rate.
Vector quantizers map a number of unique input vectors onto a smaller number of output vectors in order to reduce the bit rate. For example, a set of 100 unique input vectors might be mapped to a set of 10 output vectors that have been determined by a quantizer designer to be representative of the input vector set. The bit rate reduction achieved is that the input vector set requires a 8-bit binary code to represent all of the values and the output vector set requires only a 5-bit code. Each input vector is mapped to the output vector that is "closed" to the input vector. For example, if the input vector (2, 2) and two of the output vectors are (3, 4) and (5, 5), the input vector would be mapped to the output vector (3, 4) because it most closely represents the input vector. The goal of the mapping is to obtain a cluster of input vectors around each output vector. Each input vector is then represented by the closest output vector. The output vectors are then encoded into a binary code. When the output vectors are decoded and the decoded image assembled, the decoded image lacks the detail of the original image. The decoded image is constructed only from the output vectors, which do not correspond exactly to the original input vectors. One of the major focuses of vector quantization design is the balance between the size of the output vector set and the bit rate.
Decoding images, constructed by a low bit rate scheme, generally suffer from the following types of degradations: (a) edge regions in the decoded image contain staircase effects, (b) quasi-constant or slowly varying regions suffer from contouring effects, and (c) textured regions lose detail and suffer from granular noise. These problems arise in images encoded/decoded by both scalar and vector quantization schemes. For low bit rate coding applications, an acceptable rate of transmission lies in the range of 0.2 bit/pixel to 1 bit/pixel. At these rates, the staircase and contouring effects are pronounced and affect the visual quality of the decoded image.
Staircase and contouring effects in a decoded image are generally fairly obvious to the naked eye. The staircase effect is the blockiness that occurs at an edge region, i.e., a region wherein two distinct pixel value regions meet. A staircase effect is produced by the fact that there are fewer output vector values than input vector values and thus the intensity changes along decoded edge regions are more pronounced. Additionally, most quantizers tend to convert diagonal edge regions into staircase type regions due to the encoding schemes. The contouring effects that appear in the slowly varying regions are also be caused by the fact that not all of the variations in the intensity of the original image are available for the decoded image. For example, if a region of the original image included an area having 4 intensity changes therein, the decoded image might represent the area with only 2 intensities. In contrast to the staircase effects and the contouring effects, the effect of the granular noise on the viewed image is often mitigated by the very nature of the textured regions. However, some blockiness is still likely to be visible in these regions due to the lower bit rate, i.e., the decrease in intensity variation.
Each of the degradations represents a loss of fidelity of the decoded image from the original image. In vector quantization systems, the three common types of degradation can be traced to the finite size of the set of encoding values, the distortion measure used in the quantizer design, i.e., how close the input values must be to the encoded values, and the finite training procedure involved in the design of the quantizer. Most present systems for enhancing decoded images are directed toward modifying the operations of the quantizer or encoding system and/or modifying the bit rate.
The present invention is directed to a system for enhancing the quality of a decoded image at the image decoder without affecting the operation of the quantizer or other encoding method and without increasing the bit rate. Because the system does not affect the encoding or transmission processes, it can be readily integrated into established quantizer systems. The present invention is suitable for enhancing any decoded image whether it is a stored digital image or an image received in real time.